Image sensor combining high dynamic range techniques

ABSTRACT

Various technologies described herein pertain to combining high dynamic range techniques to enable rendering higher dynamic range scenes with an image sensor. The image sensor can implement a combination of spatial exposure multiplexing and temporal exposure multiplexing, for example. By way of another example, the image sensor can implement a combination of spatial exposure multiplexing and dual gain operation. Pursuant to another example, the image sensor can implement a combination of temporal exposure multiplexing and dual gain operation. In accordance with yet another example, the image sensor can implement a combination of spatial exposure multiplexing, temporal exposure multiplexing, and dual gain operation.

BACKGROUND

An image sensor is a device that can convert an optical image into anelectronic signal. Image sensors are oftentimes utilized in stillcameras, video cameras, video systems, and other imaging devices.Cameras and other imaging devices commonly employ either acharge-coupled device (CCD) image sensor or a complementarymetal-oxide-semiconductor (CMOS) image sensor.

CMOS image sensors include an array of pixels, each of which cancomprise a photodetector. CMOS image sensors also include circuitry toconvert light energy to an analog voltage and additional circuitry toconvert the analog voltage to digital data. A CMOS image sensor can bean integrated circuit (e.g., a system on chip (SoC)) that includesvarious analog, digital, and/or mixed-signal components for capturinglight and processing imaging related information. For example,components integrated into the CMOS image sensor oftentimes include aprocessor module (e.g., microprocessor, microcontroller, or digitalsignal processor (DSP) core), memory, analog interfaces (e.g.,analog-to-digital converters, digital-to-analog converters), and soforth.

Visible imaging systems implemented using CMOS image sensors can reducecosts, power consumption, and noise while improving resolution. Forinstance, cameras can use CMOS image sensors that efficiently marrylow-noise image detection and signal processing with multiple supportingblocks that can provide timing control, clock drivers, referencevoltages, analog to digital conversion, digital to analog conversion,key signal processing elements, and the like. High-performance videocameras can thereby be assembled using a CMOS integrated circuitsupported by few components including a lens and a battery, forinstance. Accordingly, by leveraging CMOS image sensors, camera size canbe decreased and battery life can be increased. Also, dual-use camerashave emerged that can employ CMOS image sensors to alternately producehigh-resolution still images or high definition (HD) video.

When capturing an image of a scene using an image sensor, lightingconditions can vary. For instance, the scene may be too dark, too light,too diverse, or too dynamic, such as when a cloud quickly blocks thesun. To adjust to different lighting conditions, it is desirable to havean image sensor with a wide dynamic range, where the image sensor canadjust to current lighting conditions to enhance details in the image.Yet, dynamic ranges of many conventional image sensors are oftentimesnot high enough, which leads to these image sensors being unable tocapture some details in a scene. By way of illustration, sometraditional image sensors, when capturing an image of an outdoor scenein bright light, may be unable to satisfactorily render a portion of thescene in a shadow (e.g., details of the scene in the shadow may berendered as pitch black) and/or a portion of the scene in high light(e.g., details of the scene in high light may be rendered as white) dueto the typical dynamic ranges of such traditional image sensors.

SUMMARY

Described herein are various technologies that pertain to combining highdynamic range techniques to enable rendering higher dynamic range sceneswith an image sensor. Examples of the high dynamic range techniques thatcan be combined by the image sensor include spatial exposuremultiplexing, temporal exposure multiplexing, and/or dual gainoperation. However, other high dynamic range techniques can additionallyor alternatively be supported by the image sensor. By way ofillustration, typical image sensors oftentimes output images with adynamic range between 8 and 14 bits. In contrast, by combining two ormore high dynamic range techniques as described herein, an image sensorcan allow for generating an image with a 20+ bit dynamic range, forexample. Moreover, combining the high dynamic range techniques candecrease detrimental impact caused by motion artifacts and/or decreasepower consumption as compared to some conventional approaches forenhancing dynamic range.

According to an example, the image sensor can implement a combination ofspatial exposure multiplexing and temporal exposure multiplexing. By wayof another example, the image sensor can implement a combination ofspatial exposure multiplexing and dual gain operation. Pursuant toanother example, the image sensor can implement a combination oftemporal exposure multiplexing and dual gain operation. In accordancewith yet another example, the image sensor can implement a combinationof spatial exposure multiplexing, temporal exposure multiplexing, anddual gain operation.

As described herein in accordance with various embodiments, the imagesensor includes a pixel array that comprises pixels. Moreover, the imagesensor includes a timing controller configured to control exposure timesof the pixels in the pixel array. The image sensor further includes areadout circuit configured to read out signals from the pixels in thepixel array and an analog-to-digital converter configured to convert thesignals from the pixels to pixel values for the pixels (e.g., digitalpixel values). An output frame can be generated (e.g., by an imagesignal processor included in the image sensor or a separate image signalprocessor in communication with the image sensor) based on the pixelsvalues for the pixels.

In accordance with various embodiments, the image sensor can implement acombination that includes at least spatial exposure multiplexing andtemporal exposure multiplexing. Accordingly, the timing controller canbe configured to control a first subset of the pixels in the pixel arrayto have a first exposure time during a first time period and configuredto control a second subset of the pixels in the pixel array to have asecond exposure time during the first time period, where a first frameof an input image stream can be captured during the first time period.Moreover, the timing controller can be configured to control the firstsubset of the pixels in the pixel array to have a third exposure timeduring a second time period and configured to control the second subsetof the pixels in the pixel array to have a fourth exposure time duringthe second time period, where a second frame of the input image streamcan be captured during the second time period. Further, the firstexposure time, the second exposure time, the third exposure time, andthe fourth exposure time differ from each other. The readout circuit canbe configured to read out signals from the pixels in the pixel array forthe first frame and the second frame of the input image stream.Moreover, the analog-to-digital converter can be configured to convertthe signals to pixel values for the first frame and the second frame ofthe input image stream. Further, an output frame can be generated basedon the pixel values for the first frame and the second frame of theinput image stream.

According to various embodiments, the image sensor can implement acombination that includes dual gain operation as well as at least one ofspatial exposure multiplexing or temporal exposure multiplexing. Thus,the timing controller can be configured to control the exposure times ofthe pixels at least one of temporally or spatially such that an inputimage stream has two or more differing exposure times. The readoutcircuit can be configured to read out signals from the pixels in thepixel array for the input image stream having the two or more differingexposure times. The readout circuit can further be configured to amplifythe signals read out from the pixels in the pixel array by a firstanalog gain to output first amplified signals for the input image streamhaving the two or more differing exposure times. Moreover, the readoutcircuit can be configured to amplify the signals read out from thepixels in the pixel array by a second analog gain to output secondamplified signals for the input image stream having the two or morediffering exposure times, where the first analog gain differs from thesecond analog gain. The analog-to-digital converter of the image sensorcan be configured to convert the first amplified signals to firstamplified pixel values for the input image stream having the two or morediffering exposure times. The analog-to-digital converter can further beconfigured to convert the second amplified signals to second amplifiedpixel values for the input image stream having the two or more differingexposures times. An output frame can be generated based on the firstamplified pixel values for the input image stream having the two or morediffering exposure times and the second amplified pixel values for theinput image stream having the two or more differing exposure times.

The above summary presents a simplified summary in order to provide abasic understanding of some aspects of the systems and/or methodsdiscussed herein. This summary is not an extensive overview of thesystems and/or methods discussed herein. It is not intended to identifykey/critical elements or to delineate the scope of such systems and/ormethods. Its sole purpose is to present some concepts in a simplifiedform as a prelude to the more detailed description that is presentedlater.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of an exemplary image sensor.

FIGS. 2-3 illustrate exemplary patterns of pixels in a pixel array forspatial exposure multiplexing.

FIG. 4 illustrates a block diagram of another exemplary image sensor.

FIG. 5 illustrates a block diagram of yet another exemplary imagesensor.

FIG. 6 illustrates a block diagram of an exemplary system that includesan image sensor and an image signal processor.

FIG. 7 illustrates an exemplary scenario where an image sensorimplements a combination of spatial exposure multiplexing and temporalexposure multiplexing.

FIGS. 8-10 illustrate exemplary scenarios wherein an image sensorimplements a combination of dual gain operation as well as at least oneof spatial exposure multiplexing or temporal exposure multiplexing.

FIG. 11 illustrates an exemplary CMOS image sensor pixel that can beincluded in a pixel array.

FIG. 12 is a flow diagram that illustrates an exemplary methodology ofincreasing dynamic range of an image sensor.

FIG. 13 is a flow diagram that illustrates another exemplary methodologyof increasing dynamic range of an image sensor.

FIG. 14 illustrates an exemplary computing device.

DETAILED DESCRIPTION

Various technologies pertaining to combining high dynamic rangetechniques to enable rendering higher dynamic range scenes with an imagesensor (as compared to conventional high dynamic range approaches) arenow described with reference to the drawings, wherein like referencenumerals are used to refer to like elements throughout. In the followingdescription, for purposes of explanation, numerous specific details areset forth in order to provide a thorough understanding of one or moreaspects. It may be evident, however, that such aspect(s) may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order tofacilitate describing one or more aspects. Further, it is to beunderstood that functionality that is described as being carried out bycertain system components may be performed by multiple components.Similarly, for instance, a component may be configured to performfunctionality that is described as being carried out by multiplecomponents.

Moreover, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom the context, the phrase “X employs A or B” is intended to mean anyof the natural inclusive permutations. That is, the phrase “X employs Aor B” is satisfied by any of the following instances: X employs A; Xemploys B; or X employs both A and B. In addition, the articles “a” and“an” as used in this application and the appended claims shouldgenerally be construed to mean “one or more” unless specified otherwiseor clear from the context to be directed to a singular form.

Referring now to the drawings, FIG. 1 illustrates an exemplary imagesensor 100. The image sensor 100 can be a CMOS image sensor system onchip. According to various examples, a camera, a video system, a medicalimaging device, an industrial imaging device, a microscope, or the likecan include the image sensor 100. Examples of a camera that can includethe image sensor 100 include a digital camera, a videoconference camera,a broadcast video camera, a cinematography camera, a surveillance videocamera, a handheld video camera, a camera integrated into a computingdevice, a high dynamic range implementation of a camera, and so forth.Moreover, examples of computing devices (which can include the imagesensor 100 as part of a camera) include a desktop computing device, amobile computing device (e.g., a laptop computing device, a mobiletelephone, a smartphone, a tablet computing device, a wearable computingdevice, a handheld computing device, a portable gaming device, apersonal digital assistant), a gaming console, an in-vehiclecommunications and infotainment system, or the like.

The image sensor 100 includes a pixel array 102. The pixel array 102 caninclude M rows and N columns of pixels, where M and N can be anyintegers. Each pixel in the pixel array 102 can comprise a photodetector(e.g., photogate, photoconductor, photodiode) that overlays a substrateto generate a photo-generated charge. Each pixel can also include asource follower transistor and a floating diffusion region connected toa gate of the source follower transistor. Accordingly, charge generatedby the photodetector can be sent to the floating diffusion region toconvert the charge to a voltage that is readable by transistor elementsand can be processed by signal processing circuits, either within thepixel or in other parts of the pixel array 102 (or other parts of theimage sensor 100). Further, each pixel can include a transistor fortransferring charge from the photodetector to the floating diffusionregion and another transistor for resetting the floating diffusionregion to a predetermined charge level prior to charge transference;yet, it is to be appreciated that the claimed subject matter is notlimited to the foregoing, as other pixel architectures are intended tofall within the scope of the hereto appended claims.

The image sensor 100 includes a timing controller 104 configured tocontrol exposure times of the pixels in the pixel array 102. The timingcontroller 104 can be configured to control the exposure times of thepixels temporally and/or spatially such that an input image streamcaptured by the image sensor 100 has two or more differing exposuretimes. The timing controller 104 can spatially control the exposuretimes of the pixels by controlling a first subset of the pixels in thepixel array 102 and a second subset of the pixels in the pixel array 102to have differing exposure times during a common time period. Asdescribed herein, the first subset of the pixels in the pixel array 102and the second subset of the pixels in the pixel array 102 are disjoint.Additionally or alternatively, the timing controller 104 can temporallycontrol the exposure times of the pixels by controlling the pixels inthe pixel array 102 (or a subset of such pixels) to have differingexposure times during differing time periods (e.g., during whichdiffering frames in the input image stream are captured).

As used herein, the term “exposure time” refers to a length of time. Toprovide a high dynamic range, the pixel array 102 can include pixelsthat have differing exposure times during a common time period (e.g., toenable implementing spatial exposure multiplexing). Additionally oralternatively, the pixels in the pixel array 102 (or a subset of suchpixels) can have differing exposure times during different time periods(e.g., to enable implementing temporal exposure multiplexing). Accordingto an example, the exposure times set by the timing controller 104 canbe programmatically varied, and thus, a dynamic range of the imagesensor 100 can be altered. However, pursuant to other examples, it iscontemplated that the exposure times controlled by the timing controller104 can be static.

According to an example, the timing controller 104 can employ twodistinct exposure times (e.g., a long exposure time and a short exposuretime) if one of spatial exposure multiplexing or temporal exposuremultiplexing is implemented by the image sensor 100. Following thisexample, it is contemplated that an exposure ratio of 16:1 or 32:1 maybe used by the timing controller 104, where the exposure ratio specifiesa ratio between the long exposure time and the short exposure time. Thelong exposure time can enable rendering bright parts of a scene, and theshort exposure time can enable rendering dark parts of the scene. Forinstance, a 16:1 exposure ratio can enable adding 4 bits of dynamicrange to a resulting output image (relative to an image sensor that usesa single exposure time). However, it is to be appreciated that theclaimed subject matter is not limited to the foregoing example. Pursuantto another example, the timing controller 104 can utilize four distinctexposure times if both spatial exposure multiplexing and temporalexposure multiplexing are implemented by the image sensor 100.

The image sensor 100 can also include read buses 106 and a readoutcircuit 108. The readout circuit 108 can be configured to read outsignals (e.g., voltages) from the pixels in the pixel array 102. Thesignals can be transferred from the pixels in the pixel array 102 viathe read buses 106 to the readout circuit 108. The image sensor 100 caninclude N read buses 106, where each read bus can be associated with arespective column of the pixel array 102. By way of another example,columns of the pixel array 102 can share read buses 106, and thus, theimage sensor 100 can include fewer than N read buses 106. Pursuant toyet another example, each column of the pixel array 102 can beassociated with more than one read bus, and thus, the image sensor 100can include more than N read buses 106.

The readout circuit 108 can include voltage amplifier(s) that amplifythe signals (e.g., voltages) read out from the pixels of the pixel array102. For example, the readout circuit 108 can include N voltageamplifiers (e.g., a voltage amplifier for each column of the pixel array102). According to another example, the readout circuit 108 can includefewer than N voltage amplifiers (e.g., columns of the pixel array 102can share voltage amplifiers). In accordance with yet another example,the readout circuit 108 can include more than N voltage amplifiers.

Moreover, pursuant to an example, the signals read out from the pixelsof the pixel array 102 can each be amplified by more than one voltageamplifier of the readout circuit 108 in parallel (e.g., to support dualgain operation). Following this example, a signal read out from aparticular pixel in the pixel array 102 can be amplified by two voltageamplifiers in parallel (e.g., a high gain amplifier and a low gainamplifier). Thus, the signal from the particular pixel can be processedby the readout circuit 108 in parallel through two signal chains withdifferent analog gains. Accordingly, following this example, the readoutcircuit 108 can include a plurality of signal chains for each column ofthe pixel array 102 to enable dual gain operation.

The readout circuit 108 can further include sampling capacitors.Sampling capacitors can be respectively coupled to corresponding outputsof the voltage amplifiers of the readout circuit 108. By way ofillustration, an amplified signal outputted by a particular voltageamplifier of the readout circuit 108 can be provided to a correspondingsampling capacitor.

The image sensor 100 further includes an analog-to-digital converter(ADC) 110. The analog-to-digital converter 110 can be configured toconvert the signals read out by the readout circuit 108 to pixel values(e.g., digital pixel values) for the pixels in the pixel array 102.Thus, a voltage read out from a particular pixel of the pixel array 102by the readout circuit 108 can be converted to a digital pixel value forthe particular pixel by the analog-to-digital converter 110. Accordingto another illustration, amplified signals memorized into samplingcapacitors of the readout circuit 108 can be converted by theanalog-to-digital converter 110 to corresponding pixel values. Whilemany of the examples set forth herein describe an image sensor thatincludes one analog-to-digital converter, it is contemplated that theseexamples can be extended to scenarios where a plurality ofanalog-to-digital converters are included in an image sensor, up to andincluding an analog-to-digital converter per pixel.

The image sensor 100 supports combining high dynamic range techniques toenable rendering higher dynamic range scenes as compared to conventionalapproaches. Examples of the high dynamic range techniques that can becombined by the image sensor 100 include spatial exposure multiplexing,temporal exposure multiplexing, and/or dual gain operation. Thus,multiple pixel values resulting from different exposure times and/ordifferent gains (e.g., due to a combination of high dynamic rangetechniques being implemented) can be stitched together to generate anoutput frame having increased dynamic range. While these three highdynamic range techniques are described herein, it is contemplated thatother high dynamic range techniques can additionally or alternatively besupported by the image sensor 100.

Spatial exposure multiplexing varies exposure times between differentpixels of the same frame. Thus, neighboring pixels of different exposuretimes can be stitched together to render a higher dynamic range, albeitat a loss of spatial resolution. If spatial exposure multiplexing isimplemented by the image sensor 100 in combination with at least oneother high dynamic range technique, two different exposure times can beused in the same frame to obtain a desired dynamic range (e.g., 20+ bitdynamic range). In contrast, if spatial exposure multiplexing were to beimplemented without another high dynamic range technique, then three ormore exposure times may be used in the same frame to obtain the desireddynamic range; yet, as the number of exposure times used in one frameincreases, the spatial resolution is detrimentally impacted.

Temporal exposure multiplexing utilizes two or more successive frames(e.g., snapshots) captured using different exposure times, where theframes can be stitched together into a higher dynamic range scene. Forexample, if two 12 bit images taken with a 16:1 exposure ratio arestitched together, a resulting output image can render up to 16 bits ofdynamic range. If temporal exposure multiplexing were to be implementedwithout another high dynamic range technique, then three or moresuccessive frames (each using a different exposure time) may be stitchedtogether to obtain a desired dynamic range (e.g., 20+ bit dynamicrange). However, this approach leads to increasing the frame rate atwhich an image sensor runs (e.g., the image sensor can be run threetimes faster than a resulting output video stream if three successiveframes are stitched together), which shortens the integration time foreach individual frame, increases power consumption for the image sensor,and increases a likelihood of motion artifacts being captured in theframes being combined.

Dual gain operation allows for the image sensor 100 to output two framesof data simultaneously at different analog gains. The two frames havingthe two gains can subsequently be stitched together for a higher dynamicrange image.

According to an illustration, conventional image sensors commonly outputimages with dynamic ranges between 8 and 14 bits depending on designs ofthe image sensors. These typical dynamic ranges can limit abilities ofthe conventional image sensors to render full dynamic ranges of highcontrast scenes. On the contrary, by combining two or more high dynamicrange techniques as described herein, the image sensor 100 can allow forgenerating an output image with a 20+ bit dynamic range, for example, toachieve at least 120 dB dynamic range (e.g., such dynamic range may beneeded for automotive applications).

The image sensor 100 can yield at least four stitchable fields of pixelvalues from no more than two frames of an input image stream, where thestitchable fields can be combined to generate an output frame. Accordingto an example, the image sensor 100 can implement a combination ofspatial exposure multiplexing and temporal exposure multiplexing. By wayof another example, the image sensor 100 can implement a combination ofspatial exposure multiplexing and dual gain operation. Pursuant toanother example, the image sensor 100 can implement a combination oftemporal exposure multiplexing and dual gain operation. In accordancewith yet another example, the image sensor 100 can implement acombination of spatial exposure multiplexing, temporal exposuremultiplexing, and dual gain operation.

If the image sensor 100 implements spatial exposure multiplexing,various exposure time patterns for pixels in the pixel array 102 areintended to fall within the scope of the hereto appended claims.Examples of the patterns are depicted in FIGS. 2-3. In particular, FIG.2 shows an exemplary checkerboard pattern, and FIG. 3 shows an exemplaryzigzag pattern.

FIGS. 2-3 illustrate differing examples of portions of the pixel array102. The pixel array 102 includes pixels in differing Bayer domains. Thepixel array 102 includes four types of pixels that belong to differingBayer domains. The four types of pixels include red pixels (R), greenpixels next to red pixels (G_(R)) (e.g., in the same row as the redpixels), blue pixels (B), and green pixels next to the blue pixels(G_(B)) (e.g., in the same row as the blue pixels). The red pixels (R)include photodiodes operative based upon obtaining red light. The greenpixels (G_(R)) and the green pixels (G_(B)) include photodiodes thatoperate based upon obtaining green light. Further, the blue pixels (B)include photodiodes that operate based upon obtaining blue light. Thegreen pixels, (G_(B)) and (G_(R)), are differentiated from each otherbased upon the alternating color in the respective row, and suchdifferentiation provides a manner of identifying four separate Bayerdomains.

The pixel array 102 can include substantially any number of pixels. Theportion 200 of the pixel array 102 illustrated in FIG. 2 and the portion300 of the pixel array 102 illustrated in FIG. 3 show layouts of thefour types of pixels that belong to the disparate Bayer domains that canbe utilized across the pixel array 102. However, it is to be appreciatedthat other layouts are intended to fall within the scope of the heretoappended claims.

Turning to FIG. 2, illustrated is a portion 200 of the pixel array 102that includes a first subset of the pixels having a first exposure timeand a second subset of the pixels having a second exposure time during acommon time period (e.g., during which a single contiguous frame of aninput image stream is captured), where the first exposure time differsfrom the second exposure time. The subsets of the pixels of the pixelarray 102 having the respective exposure times (e.g., integration times)are spatially arranged according to a checkerboard pattern in FIG. 2,where alternating blocks of pixels are formed. The differing exposuretimes can be controlled by the timing controller 104 as described hereinto support spatial exposure multiplexing; thus, the timing controller104 can control the first subset of the pixels in the pixel array 102 tohave the first exposure time and the second subset of the pixels in thepixel array 102 to have the second exposure time during the common timeperiod.

The first subset of the pixels in the pixel array 102 can include firstrectangular blocks of pixels (depicted as shaded boxes in FIG. 2)triggered for the first exposure time. Moreover, the second subset ofthe pixels in the pixel array 102 can include second rectangular blocksof pixels (depicted as white boxes in FIG. 2) triggered for the secondexposure time. The first rectangular blocks of pixels and the secondrectangular blocks of pixels can alternate and repeat across at least aportion of the pixel array 102 as illustrated in FIG. 2.

According to the depicted example shown in FIG. 2, each of the firstrectangular blocks of pixels and each of the second rectangular blocksof pixels can be 2×2 blocks of pixels. Each rectangular block of pixelscan include a pixel from each Bayer domain. Thus, each of the firstrectangular blocks of pixels and each of the second rectangular blocksof pixels can include a first green pixel (G_(B)), a blue pixel (B), ared pixel (R), and a second green pixel (G_(R)). The blue pixel (B) andthe first green pixel (G_(B)) can be adjacent in a row of the pixelarray 102. Moreover, the red pixel (R) and the second green pixel(G_(R)) can be adjacent in a differing row of the pixel array 102.Further, the first green pixel (G_(B)) and the red pixel (R) can beadjacent in a column of the pixel array 102, and the blue pixel (B) anda second green pixel (G_(R)) can be adjacent in a differing column ofthe pixel array 102. Further, no two rectangular blocks of the firstrectangular blocks of pixels are orthogonal in the pixel array 102.Likewise, no two rectangular blocks of the second rectangular blocks ofpixels are orthogonal in the pixel array 102.

Assuming the first exposure time differs from the second exposure time,long exposure pixels and short exposure pixels can be captured withinthe same frame. Moreover, an orthogonal checkerboard pattern of lightexposure and dark exposure areas can result. Further, each rectangularblock of pixels in the checkerboard pattern is a self-containedresolution block.

With reference to FIG. 3, illustrated is a portion 300 of the pixelarray 102 that includes a first subset of the pixels having a firstexposure time and a second subset of the pixels having a second exposuretime during a common time period (e.g., during which a single contiguousframe of an input image stream is captured), where the first exposuretime differs from the second exposure time. The subsets of the pixels ofthe pixel array 102 having the respective exposure times (e.g.,integration times) are spatially arranged according to a zigzag patternin FIG. 3. Again, the differing exposure times can be controlled by thetiming controller 104 as described herein to support spatial exposuremultiplexing; thus, the timing controller 104 can control the firstsubset of the pixels in the pixel array 102 to have the first exposuretime and the second subset of the pixels in the pixel array 102 to havethe second exposure time during the common time period.

As illustrated in FIG. 3, the subsets of the pixels of the pixel array102 having the differing exposure times are spatially arranged accordingto a zigzag pattern. Again, the first subset of the pixels in the pixelarray 102 triggered for the first exposure time can be represented asshaded boxes in FIG. 3, and the second subset of the pixels in the pixelarray 102 triggered for the second exposure time can be represented aswhite boxes in FIG. 3.

The bold line in FIG. 3 highlights an 8 pixel unit cell layout with fourpixels having the first exposure time and four pixels having the secondexposure time. Thus, the 8 pixel unit cell can include long exposurepixels and short exposure pixels. The 8 pixel unit cell can include afirst non-rectangular unit of pixels (triggered for the first exposuretime) and a second non-rectangular unit of pixels (triggered for thesecond exposure time).

Pursuant to the example depicted in FIG. 3, the first subset of thepixels in the pixel array 102 can include first non-rectangular units ofpixels and the second subset of the pixels in the pixel array 102 caninclude second non-rectangular units of pixels, where the firstnon-rectangular units of pixels oppose the second non-rectangular unitsof pixels to form the zigzag pattern in the pixel array 102. Moreover,each of the first non-rectangular units of pixels and each of the secondnon-rectangular units of pixels are L-shaped and include three adjacentpixels in a row of the pixel array 102 and a fourth pixel in an adjacentrow of the pixel array 102. Thus, as depicted, a first non-rectangularunit of pixels can include a blue pixel (B), two green pixels (two G_(R)_(S) ), and a red pixel (R). The two green pixels and the red pixel canbe adjacent in a row of the pixel array 102 (G_(R)-R-G_(R)) and the bluepixel (B) can be adjacent to one of the green pixels (G_(R)) in a columnof the pixel array 102. Moreover, a second non-rectangular unit ofpixels can include a red pixel (R), two green pixels (two G_(B) _(S) ),and a blue pixel (B), where the green pixels and the blue pixel areadjacent pixels in a row of the pixel array 102 (G_(B)-B-G_(B)) and thered pixel (R) is adjacent to one of the green pixels (G_(B)) in a columnof the pixel array 102.

It is contemplated that either the zigzag pattern as shown in FIG. 3 orthe checkerboard pattern as shown in FIG. 2 can be utilized by the imagesensor 100 if implementing spatial exposure multiplexing (e.g., one ofthe patterns can be employed). However, other alternative patterns areintended to fall within the scope of the hereto appended claims.

Referring now to FIG. 4, illustrated is another exemplary image sensor400 (e.g., the image sensor 100) according to various embodiments. Theimage sensor 400 supports dual gain operation, allowing theanalog-to-digital converter 110 to output two frames of pixel valuessimultaneously with different gains applied thereto for each frame of aninput image stream. The image sensor 400 includes the pixel array 102,the timing controller 104, the read buses 106, the readout circuit 108,and the analog-to-digital converter 110. The readout circuit 108 furtherincludes high gain amplifier(s) 402 and low gain amplifier(s) 404.

The timing controller 104 can be configured to control exposure times ofthe pixels in the pixel array 102 at least one of temporally orspatially such that an input image stream has two or more differingexposure times. Moreover, the readout circuit 108 can be configured toread out signals from the pixels in the pixel array 102 for the inputimage stream having the two or more differing exposure times. Thereadout circuit 108 can further be configured to amplify the signalsread out from the pixels in the pixel array 102 by a first analog gainto output first amplified signals for the input image stream having thetwo or more differing exposure times. Moreover, the readout circuit 108can be configured to amplify the signals read out from the pixels in thepixel array 102 by a second analog gain to output second amplifiedsignals for the input image stream having the two or more differingexposure times, where the first analog gain differs from the secondanalog gain. According to an illustration, the high gain amplifier(s)402 can amplify the signals read out from the pixels in the pixel array102 by a high analog gain (e.g., the first analog gain), and the lowgain amplifier(s) 404 can amplify the signals read out from the pixelsin the pixel array 102 by a low analog gain (e.g., the second analoggain). The low analog gain of the low gain amplifier(s) 404 can be tunedto enable rendering a full dynamic range of a pixel (e.g., all of thecharge that the pixel is capable of collecting), whereas the high analoggain of the high gain amplifier(s) 402 can be tuned to be noise limitedby the pixel (e.g., tuned to render a smallest possible signal that thepixel can sense).

The analog-to-digital converter 110 can further be configured to convertthe first amplified signals to first amplified pixel values for theinput image stream having the two or more differing exposure times.Moreover, the analog-to-digital converter 110 can be configured toconvert the second amplified signals to second amplified pixel valuesfor the input image stream having the two or more differing exposuretimes.

Now turning to FIG. 5, illustrated is another exemplary image sensor 500(e.g., the image sensor 100) pursuant to various embodiments. Again, theimage sensor 500 includes the pixel array 102, the timing controller104, the read buses 106, the readout circuit 108, and theanalog-to-digital converter 110. Although not shown, it is contemplatedthat the readout circuit 108 can include the high gain amplifier(s) 402and the low gain amplifier(s) 404.

The image sensor 500 further includes an image signal processor 502. Theimage signal processor 502 can be configured to generate an output framebased on the pixel values outputted by the analog-to-digital converter110. For instance, if the image sensor 500 employs temporal exposuremultiplexing and spatial exposure multiplexing, then the image signalprocessor 502 can generate the output frame based on pixel values fortwo frames (e.g., a first frame and a second frame, successive frames)in an input image stream. According to another example, if the imagesensor 500 employs dual gain operation, the image signal processor 502can generate the output frame based on first amplified pixel values(e.g., amplified by the high gain amplifier(s) 402) for the input imagestream having the two or more differing exposure times and secondamplified pixel values (e.g., amplified by the low gain amplifier(s)404) for the input image stream having the two or more differingexposure times. Following this example, the image signal processor 502can generate the output frame based on the first amplified pixel valuesand the second amplified pixels values for one frame of the input imagestream (assuming temporal exposure multiplexing is not employed) or twoframes of the input image stream (assuming temporal exposuremultiplexing is employed).

With reference to FIG. 6, illustrated is an exemplary system 600 thatincludes an image sensor 602 (e.g., the image sensor 100) and an imagesignal processor 604. The image sensor 602 includes the pixel array 102,the timing controller 104, the read buses 106, the readout circuit 108,and the analog-to-digital converter 110. Moreover, it is contemplatedthat the readout circuit 108 can include the high gain amplifier(s) 402and the low gain amplifier(s) 404.

In the example depicted in FIG. 6, the image signal processor 604 isseparate from the image sensor 602, with the image sensor 602 and theimage signal processor 604 being in communication with each other. Theimage sensor 602 can be configured to output pixel values (e.g., fromthe analog-to-digital converter 110) to the image signal processor 604.Moreover, an output frame can be generated by the image signal processor604 based on the pixel values obtained from the image sensor 602;accordingly, similar to the image signal processor 502 of FIG. 5, theimage signal processor 604 can generate the output frame.

According to an illustration where a combination of at least spatialexposure multiplexing and temporal exposure multiplexing is employed,the image sensor 602 can output the pixel values for the first frame andthe second frame in the input image stream to the image signal processor604. Accordingly, an output frame can be generated by the image signalprocessor 604 based on the pixel values for the first frame and thesecond frame of the input image stream.

Pursuant to another illustration where dual gain operation is employed,the image sensor 602 can output the first amplified pixel values for theinput image stream having the two or more differing exposure times andthe second amplified pixel values for the input image stream having thetwo or more differing exposure times. Further, the image signalprocessor 604 can generate an output frame based on the first amplifiedpixel values and the second amplified pixel values.

FIG. 7 depicts an exemplary scenario 700 where an image sensor (e.g.,the image sensor 100, the image sensor 500, the image sensor 602)implements a combination of spatial exposure multiplexing and temporalexposure multiplexing. Following this exemplary scenario 700, the timingcontroller 104 of the image sensor can control the exposure times of thepixels 102 both temporally and spatially. More particularly, the timingcontroller 104 can control a first subset of the pixels in the pixelarray 102 to have a first exposure time during a first time period.Moreover, the timing controller 104 can control a second subset of thepixels in the pixel array 102 to have a second exposure time during thefirst time period. A first frame (frame i) of an input image stream canbe captured during the first time period. Further, the timing controller104 can control the first subset of the pixels in the pixel array 102 tohave a third exposure time during a second time period, and the timingcontroller 104 can control the second subset of the pixels in the pixelarray 102 to have a fourth exposure time during the second time period.A second frame (frame i+1) of the input image stream can be capturedduring the second time period. Following this example, the firstexposure time, the second exposure time, the third exposure time, andthe fourth exposure time differ from each other.

The readout circuit 108 can read out signals from the pixels in thepixel array 102 for the first frame and the second frame of the inputimage stream. Further, the analog-to-digital converter 110 can convertthe signals to pixel values for the first frame and the second frame ofthe input image stream. Moreover, an image signal processor (e.g., theimage signal processor 502, the image signal processor 604) can generatean output frame (output frame j) based on the pixel values for the firstframe (frame i) and the second frame (frame i+1) of the input imagestream.

It is contemplated that other output frames of an output image streamcan similarly be generated (e.g., an output frame j+1 can be generatedbased on pixel values for frame i+2 and frame i+3). However, it is to beappreciated that a single output frame (output frame j) may be generatedas opposed to an output image stream.

FIGS. 8-10 illustrate exemplary scenarios wherein an image sensor (e.g.,the image sensor 100, image sensor 400, the image sensor 500, the imagesensor 602) implements a combination of dual gain operation as well asat least one of spatial exposure multiplexing or temporal exposuremultiplexing. The timing controller 104 of the image sensor can controlthe exposure times of the pixels at least one of temporally or spatiallysuch that an input image stream has two or more differing exposuretimes. Moreover, the readout circuit 108 can read out signals from thepixels in the pixel array 102 for the input image stream having the twoor more differing exposure times. The readout circuit 108 (e.g., thehigh gain amplifier(s) 402) can further amplify the signals read outfrom the pixels in the pixel array 102 by a first analog gain to outputfirst amplified signals for the input image stream having the two ormore differing exposure times. Moreover, the readout circuit 108 (e.g.,the low gain amplifier(s) 404) can amplify the signals read out from thepixels in the pixel array 102 by a second analog gain to output secondamplified signals for the input image stream having the two or morediffering exposure times, where the first analog gain differs from thesecond analog gain. The analog-to-digital converter 110 of the imagesensor can convert the first amplified signals to first amplified pixelvalues for the input image stream having the two or more differingexposure times. The analog-to-digital converter 110 can further convertthe second amplified signals to second amplified pixel values for theinput image stream having the two or more differing exposures times. Anoutput frame can be generated by an image signal processor (e.g., theimage signal processor 502, the image signal processor 604) based on thefirst amplified pixel values for the input image stream having the twoor more differing exposure times and the second amplified pixel valuesfor the input image stream having the two or more differing exposuretimes.

Referring now to FIG. 8, illustrated is an exemplary scenario 800 wherethe image sensor implements a combination of spatial exposuremultiplexing and dual gain operation. Accordingly, the timing controller104 of the image sensor can control a first subset of the pixels in thepixel array 102 to have a first exposure time during a first timeperiod. Moreover, the timing controller 104 can control a second subsetof the pixels in the pixel array 102 to have a second exposure timeduring the first time period. A first frame (frame i) of an input imagestream can be captured during the first time period. Moreover, the firstexposure time and the second exposure time differ from each other.

The readout circuit 108 can read out signals from the pixels in thepixel array 102 for the first frame of the input image stream. Further,the readout circuit 108 can amplify the signals read out from the pixelsby a first analog gain to output first amplified signals, and canamplify the signals read out from the pixels by a second analog gain tooutput second amplified signals. The analog-to-digital converter 110 canconvert the first amplified signals to first amplified pixel values, andconvert the second amplified signals to second amplified pixel values.The image signal processor can further generate an output frame (outputframe j) based on the first amplified pixel values and the secondamplified pixel values for the first frame (frame i) of the input imagestream.

Other output frames of an output image stream can similarly be generated(e.g., an output frame j+1 can be generated based on first amplifiedpixel values and second amplified pixel values for frame i+1). However,it is to be appreciated that a single output frame (output frame j) maybe generated as opposed to an output image stream.

Now turning to FIG. 9, illustrated is an exemplary scenario 900 wherethe image sensor implements a combination of temporal exposuremultiplexing and dual gain operation. The timing controller 104 of theimage sensor can control the pixels in the pixel array 102 to have afirst exposure time during a first time period, where a first frame(frame i) of an input image stream is captured during the first timeperiod. Moreover, the timing controller 104 can control the pixels inthe pixel array 102 to have a second exposure time during a second timeperiod, where a second frame (frame i+1) of the input image stream iscaptured during the second time period. Again, the first exposure timeand the second exposure time differ from each other.

The readout circuit 108 can read out signals from the pixels in thepixel array 102 for the first frame of the input image stream, andamplify the signals read out for the first frame by both the firstanalog gain and the second analog gain in parallel to respectivelyoutput first amplified signals and second amplified signals for thefirst frame. The analog-to-digital converter 110 can convert the firstamplified signals to first amplified pixels values for the first frame,and convert the second amplified signals to second amplified pixelvalues for the first frame. The readout circuit 108 can similarly readout signals from the pixels in the pixel array 102 for the second frameof the input image stream, and amplify the signals by both the firstanalog gain and the second analog gain in parallel. Likewise, theanalog-to-digital converter 110 can output first amplified pixel valuesfor the second frame and second amplified pixel values for the secondframe. Moreover, the image signal processor can generate an output frame(output frame j) based on the first amplified pixel values and thesecond amplified pixel values for the first frame (frame i) of the inputimage stream and the first amplified pixel values and the secondamplified pixel values for the second frame (frame i+1) of the inputimage stream.

It is contemplated that other output frames of an output image streamcan similarly be generated (e.g., an output frame j+1 can be generatedbased on first amplified pixel values and second amplified pixel valuesfor frame i+2 and first amplified pixel values and second amplifiedpixel values for frame i+3). Yet, again, it is to be appreciated that asingle output frame (output frame j) may be generated rather than anoutput image stream.

With reference to FIG. 10, illustrated is an exemplary scenario 1000where the image sensor implements a combination of spatial exposuremultiplexing, temporal exposure multiplexing, and dual gain operation.Similar to the example set forth in FIG. 7, the timing controller 104 ofthe image sensor can control the exposure times of the pixels 102 bothtemporally and spatially. Moreover, similar to the examples described inFIGS. 8-9, the signals read out from the pixels in the pixel array 102can be amplified by two analog gains in parallel. Accordingly, firstamplified pixel values and second amplified pixel values for a firstframe (frame i) and first amplified pixel values and second amplifiedpixel values for a second frame (frame i+1) can be stitched together bythe image signal processor to generate an output frame (output frame j).

Now turning to FIG. 11, illustrated is an exemplary CMOS image sensorpixel 1100 that can be included in the pixel array 102. The pixel 1100as depicted is a 4 T pixel cell. The pixel 1100 includes a photodiode1102 connected to a transfer transistor 1104. The transfer transistor1104 is further connected to a floating diffusion region 1106. Thefloating diffusion region 1106 connects to a source follower transistor1108 and a reset transistor 1110. The source follower transistor 1108 isfurther connected to a select transistor 1112. The select transistor1112 can be employed to select a particular row of pixel cells from thepixel array 102. For instance, a select signal can be received at a gateof the select transistor 1112 to read out a value from the floatingdiffusion region 1106.

The photodiode 1102 can be charged by converting optical energy toelectrical energy. For instance, the photodiode 1102 can havesensitivity to a particular type of incident light (e.g., red light,blue light, green light). Yet, it is to be appreciated that the claimedsubject matter is not so limited.

According to an illustration, the floating diffusion region 1106 can bereset to a known state before transfer of charge to it. Resetting of thefloating diffusion region 1106 can be effectuated by the resettransistor 1110. For example, a reset signal can be received at a gateof the reset transistor 1110 to cause resetting of the floatingdiffusion region 1106. Further, the transfer transistor 1104 cantransfer charge (e.g., provided by the photodiode 1102) to the floatingdiffusion region 1106. The charge can be transferred based upon atransfer signal (TX) received at a gate of the transfer transistor 1104.Light can be integrated at the photodiode 1102 and electrons generatedfrom the light can be transferred to the floating diffusion region 1106(e.g., in a noiseless or substantially noiseless manner) when the TX isreceived at the transfer transistor 1104. Moreover, the pixel 1100(along with other pixel(s) in the same row of the pixel array 102) canbe selected for readout by employing the select transistor 1112. Readoutcan be effectuated via a read bus 1114. Further, the source followertransistor 1108 can output and/or amplify a signal representing a resetvoltage (e.g., provided via a reset bus) and a pixel signal voltagebased on the photo converted charges.

It is to be appreciated, however, that different pixel configurationsother than the example illustrated in FIG. 11 are intended to fallwithin the scope of the hereto appended claims. For instance, adisparate pixel configuration can lack the transfer gate transistor 1104(e.g., a 3 T pixel). According to another illustration, a differingpixel configuration can include more than four transistors. Yet, it isto be appreciated that the claimed subject matter is not limited to theaforementioned examples.

FIGS. 12-13 illustrate exemplary methodologies relating to increasingdynamic range of an image sensor. While the methodology is shown anddescribed as being a series of acts that are performed in a sequence, itis to be understood and appreciated that the methodology is not limitedby the order of the sequence. For example, some acts can occur in adifferent order than what is described herein. In addition, an act canoccur concurrently with another act. Further, in some instances, not allacts may be required to implement a methodology described herein.

The acts described herein may be implemented by an image sensor or animage signal processor. Moreover, the acts described herein may becomputer-executable instructions that can be implemented by one or moreprocessors and/or stored on a computer-readable medium or media. Thecomputer-executable instructions can include a routine, a sub-routine,programs, a thread of execution, and/or the like. Still further, resultsof acts of the methodology can be stored in a computer-readable medium,displayed on a display device, and/or the like.

FIG. 12 illustrates a methodology 1200 of increasing dynamic range of animage sensor. The image sensor includes a pixel array, and the pixelarray includes pixels. At 1202, a first subset of the pixels in thepixel array can be controlled to have a first exposure time during afirst time period, where a first frame of an input image stream iscaptured during the first time period. At 1204, a second subset of thepixels in the pixel array can be controlled to have a second exposuretime during the first time period. At 1206, signals can be read out fromthe pixels in the pixel array for the first frame of the input imagestream. At 1208, the signals for the first frame can be converted todigital pixel values for the first frame of the input image stream. At1210, the first subset of the pixels in the pixel array can becontrolled to have a third exposure time during a second time period,where a second frame of the input image stream is captured during thesecond time period. At 1212, the second subset of the pixels in thepixel array can be controlled to have a fourth exposure time during thesecond time period. At 1214, signals can be read out from the pixels inthe pixel array for the second frame of the input image stream. At 1216,the signals for the second frame can be converted to digital pixelvalues for the second frame of the input image stream. Moreover, anoutput frame can be generated based on the digital pixel values for thefirst frame of the input image stream and the digital pixel values forthe second frame of the input image stream.

Turning to FIG. 13, illustrated is another methodology 1300 ofincreasing the dynamic range of the image sensor. At 1302, exposuretimes of the pixels in the pixel array can be controlled at least one oftemporally or spatially. Accordingly, an input image stream has two ormore differing exposure times. At 1304, signals can be read out from thepixels in the pixel array for the input image stream having the two ormore differing exposure times. The methodology 1300 can continue inparallel to 1306 and 1310. More particularly, at 1306, the signals readout from the pixels in the pixel array can be amplified by a firstanalog gain to output first amplified signals for the input image streamhaving the two or more differing exposure times. At 1308, the firstamplified signals can be converted to first amplified digital pixelvalues for the input image stream having the two or more differingexposure times. At 1310, the signals read out from the pixels in thepixel array can be amplified by a second analog gain to output secondamplified signals for the input image stream having the two or morediffering exposure times. At 1312, the second amplified signals can beconverted to second amplified digital pixel values for the input imagestream having the two or more differing exposure times. Moreover, anoutput frame can be generated based on the first amplified digital pixelvalues for the input image stream having the two or more differingexposure times and the second amplified digital pixel values for theinput image stream having the two or more differing exposure times.

Referring now to FIG. 14, a high-level illustration of an exemplarycomputing device 1400 that can be used in accordance with the systemsand methodologies disclosed herein is illustrated. The computing device1400 may include an image sensor 1402 (e.g., the image sensor 100, theimage sensor 400, the image sensor 500, the image sensor 602). Forinstance, the image sensor 1402 can be part of a camera of the computingdevice 1400. However, according to other examples, it is contemplatedthat the image sensor 1402 can be separate from the computing device1400. The computing device 1400 further includes at least one processor1404 that executes instructions that are stored in memory 1406. Theprocessor 1404 may access the memory 1406 by way of a system bus 1408.

The computing device 1400 additionally includes a data store 1410 thatis accessible by the processor 1404 by way of the system bus 1408. Thedata store 1410 may include executable instructions, etc. The computingdevice 1400 also includes an input interface 1412 that allows externaldevices to communicate with the computing device 1400. For instance, theinput interface 1412 may be used to receive instructions from anexternal computer device, from a user, etc. The computing device 1400also includes an output interface 1414 that interfaces the computingdevice 1400 with one or more external devices. For example, thecomputing device 1400 may display text, images, etc. by way of theoutput interface 1414.

Additionally, while illustrated as a single system, it is to beunderstood that the computing device 1400 may be a distributed system.Thus, for instance, several devices may be in communication by way of anetwork connection and may collectively perform tasks described as beingperformed by the computing device 1400.

As used herein, the terms “component” and “system” are intended toencompass computer-readable data storage that is configured withcomputer-executable instructions that cause certain functionality to beperformed when executed by a processor. The computer-executableinstructions may include a routine, a function, or the like. It is alsoto be understood that a component or system may be localized on a singledevice or distributed across several devices.

Further, as used herein, the term “exemplary” is intended to mean“serving as an illustration or example of something.”

Various functions described herein can be implemented in hardware,software, or any combination thereof. If implemented in software, thefunctions can be stored on or transmitted over as one or moreinstructions or code on a computer-readable medium. Computer-readablemedia includes computer-readable storage media. A computer-readablestorage media can be any available storage media that can be accessed bya computer. By way of example, and not limitation, suchcomputer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other medium that can be used to store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer. Disk and disc, as used herein, includecompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and blu-ray disc (BD), where disks usually reproducedata magnetically and discs usually reproduce data optically withlasers. Further, a propagated signal is not included within the scope ofcomputer-readable storage media. Computer-readable media also includescommunication media including any medium that facilitates transfer of acomputer program from one place to another. A connection, for instance,can be a communication medium. For example, if the software istransmitted from a website, server, or other remote source using acoaxial cable, fiber optic cable, twisted pair, digital subscriber line(DSL), or wireless technologies such as infrared, radio, and microwave,then the coaxial cable, fiber optic cable, twisted pair, DSL, orwireless technologies such as infrared, radio and microwave are includedin the definition of communication medium. Combinations of the aboveshould also be included within the scope of computer-readable media.

What has been described above includes examples of one or moreembodiments. It is, of course, not possible to describe everyconceivable modification and alteration of the above devices ormethodologies for purposes of describing the aforementioned aspects, butone of ordinary skill in the art can recognize that many furthermodifications and permutations of various aspects are possible.Accordingly, the described aspects are intended to embrace all suchalterations, modifications, and variations that fall within the spiritand scope of the appended claims. Furthermore, to the extent that theterm “includes” is used in either the details description or the claims,such term is intended to be inclusive in a manner similar to the term“comprising” as “comprising” is interpreted when employed as atransitional word in a claim.

What is claimed is:
 1. An image sensor, comprising: a pixel array that comprises pixels; a timing controller configured to control exposure times of the pixels in the pixel array, the timing controller configured to: control a first subset of the pixels in the pixel array to have a first exposure time during a first time period, wherein a first frame of an input image stream is captured during the first time period; control a second subset of the pixels in the pixel array to have a second exposure time during the first time period; control the first subset of the pixels in the pixel array to have a third exposure time during a second time period, wherein a second frame of the input image stream is captured during the second time period; and control the second subset of the pixels in the pixel array to have a fourth exposure time during the second time period; wherein the first exposure time, the second exposure time, the third exposure time, and the fourth exposure time differ from each other; a readout circuit configured to read out signals from the pixels in the pixel array for the first frame and the second frame of the input image stream; and an analog-to-digital converter configured to convert the signals to pixel values for the first frame and the second frame of the input image stream; wherein the first subset of the pixels in the pixel array comprise first non-rectangular units of pixels; wherein the second subset of the pixels in the pixel array comprise second non-rectangular units of pixels; wherein the first non-rectangular units of pixels oppose the second non-rectangular units of pixels to form a zigzag pattern in the pixel array; and wherein each of the first non-rectangular units of pixels and each of the second non-rectangular units of pixels are L-shaped and comprise three adjacent pixels in a row of the pixel array and a fourth pixel in an adjacent row of the pixel array.
 2. The image sensor of claim 1, further comprising: an image signal processor configured to generate an output frame based on the pixel values for the first frame and the second frame of the input image stream.
 3. The image sensor of claim 1, wherein: the image sensor is configured to output the pixel values for the first frame and the second frame of the input image stream to an image signal processor; and an output frame is generated by an image signal processor based on the pixel values for the first frame and the second frame of the input image stream.
 4. The image sensor of claim 1, wherein: the readout circuit is further configured to: amplify the signals read out from the pixels in the pixel array by a first analog gain to output first amplified signals for the first frame and the second frame of the input image stream; and amplify the signals read out from the pixels in the pixel array by a second analog gain to output second amplified signals for the first frame and the second frame of the input image stream, wherein the first analog gain differs from the second analog gain; and the analog-to-digital converter is further configured to: convert the first amplified signals to first amplified pixel values for the first frame and the second frame of the input image stream; and convert the second amplified signals to second amplified pixel values for the first frame and the second frame of the input image stream.
 5. The image sensor of claim 4, further comprising: an image signal processor configured to generate an output frame based on: the first amplified pixel values for the first frame and the second frame of the input image stream; and the second amplified pixel values for the first frame and the second frame of the input image stream.
 6. The image sensor of claim 4, wherein: the image sensor is configured to output: the first amplified pixel values for the first frame and the second frame of the input image stream; and the second amplified pixel values for the first frame and the second frame of the input image stream; and an output frame is generated by an image signal processor based on: the first amplified pixel values for the first frame and the second frame of the input image stream; and the second amplified pixel values for the first frame and the second frame of the input image stream.
 7. An image sensor, comprising: a pixel array that comprises pixels, wherein the pixel array comprises a first subset of the pixels and a second subset of the pixels, wherein the first subset of the pixels in the pixel array comprise first non-rectangular units of pixels, wherein the second subset of the pixels in the pixel array comprise second non-rectangular units of pixels, wherein the first non-rectangular units of pixels oppose the second non-rectangular units of pixels to form a zigzag pattern in the pixel array, and wherein each of the first non-rectangular units of pixels and each of the second non-rectangular units of pixels are L-shaped and comprise three adjacent pixels in a row of the pixel array and a fourth pixel in an adjacent row of the pixel array; a timing controller configured to control exposure times of the pixels in the pixel array, the timing controller configured to control the exposure times of the pixels at least one of temporally or spatially such that an input image stream has two or more differing exposure times; a readout circuit configured to: read out signals from the pixels in the pixel array for the input image stream having the two or more differing exposure times; amplify the signals read out from the pixels in the pixel array by a first analog gain to output first amplified signals for the input image stream having the two or more differing exposure times; and amplify the signals read out from the pixels in the pixel array by a second analog gain to output second amplified signals for the input image stream having the two or more differing exposure times, wherein the first analog gain differs from the second analog gain; and an analog-to-digital converter configured to: convert the first amplified signals to first amplified pixel values for the input image stream having the two or more differing exposure times; and convert the second amplified signals to second amplified pixel values for the input image stream having the two or more differing exposure times.
 8. The image sensor of claim 7, the timing controller being configured to control the exposure times of the pixels temporally comprises the timing controller being configured to: control the pixels in the pixel array to have a first exposure time during a first time period, wherein a first frame of the input image stream is captured during the first time period; and control the pixels in the pixel array to have a second exposure time during a second time period, wherein a second frame of the input image stream is captured during the second time period; wherein the first exposure time and the second exposure time differ from each other.
 9. The image sensor of claim 7, the timing controller being configured to control the exposure times of the pixels spatially comprises the timing controller being configured to: control the first subset of the pixels in the pixel array to have a first exposure time during a time period, wherein a frame of the input image stream is captured during the time period; and control the second subset of the pixels in the pixel array to have a second exposure time during the time period; and wherein the first exposure time and the second exposure time differ from each other.
 10. The image sensor of claim 7, the timing controller being configured to control the exposure times of the pixels both temporally and spatially comprises the timing controller being configured to: control the first subset of the pixels in the pixel array to have a first exposure time during a first time period, wherein a first frame of the input image stream is captured during the first time period; control the second subset of the pixels in the pixel array to have a second exposure time during the first time period; control the first subset of the pixels in the pixel array to have a third exposure time during a second time period, wherein a second frame of the input image stream is captured during the second time period; and control the second subset of the pixels in the pixel array to have a fourth exposure time during the second time period; wherein the first exposure time, the second exposure time, the third exposure time, and the fourth exposure time differ from each other.
 11. The image sensor of claim 7, further comprising: an image signal processor configured to generate an output frame based on: the first amplified pixel values for the input image stream having the two or more differing exposure times; and the second amplified pixel values for the input image stream having the two or more differing exposure times.
 12. The image sensor of claim 7, wherein: an output frame is generated by an image signal processor based on: the first amplified pixel values for the input image stream having the two or more differing exposure times; and the second amplified pixel values for the input image stream having the two or more differing exposure times.
 13. A method of increasing dynamic range of an image sensor, the image sensor comprising a pixel array that comprises pixels, the method comprising: controlling a first subset of the pixels in the pixel array to have a first exposure time during a first time period, wherein a first frame of an input image stream is captured during the first time period; controlling a second subset of the pixels in the pixel array to have a second exposure time during the first time period; reading out signals from the pixels in the pixel array for the first frame of the input image stream; converting the signals for the first frame to digital pixel values for the first frame of the input image stream; controlling the first subset of the pixels in the pixel array to have a third exposure time during a second time period, wherein a second frame of the input image stream is captured during the second time period; controlling the second subset of the pixels in the pixel array to have a fourth exposure time during the second time period, wherein the first exposure time, the second exposure time, the third exposure time, and the fourth exposure time differ from each other; reading out signals from the pixels in the pixel array for the second frame of the input image stream; and converting the signals for the second frame to digital pixel values for the second frame of the input image stream; wherein the first subset of the pixels in the pixel array comprise first non-rectangular units of pixels; wherein the second subset of the pixels in the pixel array comprise second non-rectangular units of pixels; wherein the first non-rectangular units of pixels oppose the second non-rectangular units of pixels to form a zigzag pattern in the pixel array; and wherein each of the first non-rectangular units of pixels and each of the second non-rectangular units of pixels are L-shaped and comprise three adjacent pixels in a row of the pixel array and a fourth pixel in an adjacent row of the pixel array.
 14. The method of claim 13, further comprising: generating an output frame based on the digital pixel values for the first frame of the input image stream and the digital pixel values for the second frame of the input image stream. 